Augmented reality spectroscopy

ABSTRACT

In some embodiments, a system comprises a head-mounted frame removably coupleable to the user&#39;s head; one or more light sources coupled to the head-mounted frame and configured to emit light with at least two different wavelengths toward a target object in an irradiation field of view of the light sources; one or more electromagnetic radiation detectors coupled to the head-mounted member and configured to receive light reflected after encountering the target object; and a controller operatively coupled to the one or more light sources and detectors and configured to determine and display an output indicating the identity or property of the target object as determined by the light properties measured by the detectors in relation to the light properties emitted by the light sources.

PRIORITY CLAIM

This application is a continuation of U.S. application Ser. No.17/391,889, filed on Aug. 2, 2021, which is a continuation of U.S.application Ser. No. 16/775,123, filed on Jan. 28, 2020, which is acontinuation of U.S. application Ser. No. 15/713,420, filed on Sep. 22,2017, which claims the benefit of priority of U.S. ProvisionalApplication No. 62/398,454, filed on Sep. 22, 2016, all of which areincorporated herein by reference.

INCORPORATION BY REFERENCE

This application incorporates by reference the entireties of each of thefollowing US patent applications: U.S. patent application Ser. No.15/072,341; U.S. patent application Ser. No. 14/690,401; U.S. patentapplication Ser. No. 14/555,858; U.S. application Ser. No. 14/555,585;U.S. patent application Ser. No. 13/663,466; U.S. patent applicationSer. No. 13/684,489; U.S. patent application Ser. No. 14/205,126; U.S.patent application Ser. No. 14/641,376; U.S. patent application Ser. No.14/212,961; U.S. Provisional Patent Application No. 62/298,993(corresponding to U.S. patent application Ser. No. 15/425,837); and U.S.patent application Ser. No. 15/425,837.

BACKGROUND Field of the Invention

The present disclosure relates to systems and methods for augmentedreality using wearable componentry, and more specifically toconfigurations of augmented reality systems for identifying material byreflective light properties.

Description of the Related Art

Modern computing and display technologies have facilitated thedevelopment of systems for so called “virtual reality” or “augmentedreality” experiences, wherein digitally reproduced images or portionsthereof are presented to a user in a manner wherein they seem to be, ormay be perceived as, real. A virtual reality, or “VR”, scenariotypically involves presentation of digital or virtual image informationwithout transparency to other actual real-world visual input; and anaugmented reality or “AR” scenario typically involves presentation ofdigital or virtual image information as an augmentation to visualizationof the actual world around the user while still permitting the user tosubstantially perceive and view the real world.

For example, referring to FIG. 1 , an augmented reality scene (4) isdepicted wherein a user of an AR technology sees a real-world park-likesetting (6) featuring people, trees, buildings in the background, and aconcrete platform (1120). In addition to these items, the user of the ARtechnology also perceives that he “sees” a robot statue (1110) standingupon the real-world platform (1120), and a cartoon-like avatar character(2) flying by which seems to be a personification of a bumble bee, eventhough these elements (2, 1110) do not exist in the real world. As itturns out, the human visual perception system is very complex, andproducing a VR or AR technology that facilitates a comfortable,natural-feeling, rich presentation of virtual image elements amongstother virtual or real-world imagery elements is challenging. Forinstance, head-worn AR displays (or helmet-mounted displays, or smartglasses) typically are at least loosely coupled to a user's head, andthus move when the user's head moves. If the user's head motions aredetected by the display system, the data being displayed can be updatedto take the change in head pose into account. Certain aspects ofsuitable AR systems are disclosed, for example, in U.S. patentapplication Ser. No. 14/205,126, entitled “System and method foraugmented and virtual reality,” which is incorporated by reference inits entirety herein, along with the following additional disclosures,which relate to augmented and virtual reality systems such as thosedeveloped by Magic Leap, Inc. of Fort Lauderdale, Fla.: U.S. patentapplication Ser. No. 14/641,376; U.S. patent application Ser. No.14/555,585; U.S. patent application Ser. No. 14/212,961; U.S. patentapplication Ser. No. 14/690,401; U.S. patent application Ser. No.13/663,466; U.S. patent application Ser. No. 13/684,489; and U.S. PatentApplication Ser. No. 62/298,993, each of which is incorporated byreference herein in its entirety.

Systems and methods disclosed herein address various challenges anddevelopments related to AR and VR technology.

SUMMARY

A mixed reality system is configured to perform spectroscopy. Mixedreality (alternatively abbreviated as “MR”) typically involves virtualobjects integrated into and responsive to the natural world. Forexample, in an MR scenario, AR content by be occluded by real worldobjects and/or be perceived as interacting with other objects (virtualor real) in the real world. Throughout this disclosure, reference to AR,VR or MR is not limiting on the invention and the techniques may beapplied to any context.

Some embodiments are directed to a wearable system for identifyingsubstances (such as tissue, cells within tissue, or properties withincells/tissue) as a function of light wavelength emitted from andsubsequently received by/reflected to/detected at a head-mounted memberremovably coupleable to a user's head. Though this disclosure mainlyreferences tissue, or tissue properties, as a subject for analysisaccording to various embodiments, the technologies and techniques andcomponents are not limited to such. Some embodiments utilize one or morelight sources, such as electromagnetic radiation emitters coupled to thehead-mounted member, to emit light in one or more wavelengths in auser-selected direction. Such embodiments permit continuous, and evenpassive, measurements. For example, a user wearing a head mounted systemcould conduct a given activity, but inward facing sensors could detectproperties of the eye without interfering with the activity.

For example, a user could wear a system configured to look inward to theuser's eyes and identify or measure tissue properties of the eye, suchas blood concentration in a blood vessel of the eye. In other examplesof inward systems, fluids such as intraocular fluid may be analyzed andnot simply tissue properties. In other examples, a system could comprisesensors that look outward towards the external world and identify ormeasure tissue or material properties other than the eye, such as anextremity of the user or object in the ambient environment apart fromthe user.

In outward looking systems, eye tracking cameras coupled to thehead-mounted member can determine the directional gaze a user islooking, and a processor or controller may correlate that gaze withobservation of a real world target object through images captured from areal-world capturing system (such as cameras or depth sensors) coupledto the head-mounted member. Light sources coupled to the head-mountedsystem emit light away from the user, such as infrared light for examplefrom an electromagnetic radiation emitter, and in some embodiments emitlight to create an irradiation pattern in a substantially same directionas a gaze direction determined by the eye tracking cameras, therebyemitting upon the target object.

In some embodiments, real world capturing systems capture an object. Forexample a depth sensor, such as a vertical cavity surface emittinglaser, may determine the outline of an object through collecting time offlight signals impacting the object. The object, once identified at itscontours by such real-world capturing system may be highlighted andavailable for labeling. In some embodiments, a camera system of a givenfield of view defines an area available for highlighting and labelling.For example, a camera correlating to a user's gaze may encompass a 5degree field of view, 10 degree field of view, or suitable incrementspreferably up to a 30 degree central vision field of view that the lightsource will emit light substantially within.

In some embodiments, such a system further comprises one or moreelectromagnetic radiation detectors or photodetectors coupled to thehead-mounted member configured to receive reflected light that wasemitted from the light source and reflected from the target object; anda controller operatively coupled to the one or more electromagneticradiation emitters and one or more electromagnetic radiation detectorsconfigured to cause the one or more electromagnetic radiation emittersto emit pulses of light while also causing the one or moreelectromagnetic radiation detectors to detect levels of light absorptionrelated to the emitted pulses of light as a function of any receivedreflected light of a particular pulse emission.

In some embodiments, the system further comprises a processor to match awavelength of reflected light received by a detector from the targetobject to a particular material, tissue type or property of anunderlying tissue. In some embodiments other light characteristics aredetermined, such as polarization changes relative to emitted light anddetected light or scattering effects, though for purposes of thisdescription wavelength characteristics are used as an exemplary lightcharacteristic. For example, in some embodiments, an inwardelectromagnetic radiation emitter emits light in the infrared spectrumto the retina of a user, receives reflected light, and matches thewavelength of the reflected light to determine a physical property suchas the type of tissue or oxygen saturation in the tissue. In someembodiments, the system comprises outward facing light sources, andemits infrared light to a target object (such as an extremity of a useror third person), receives reflected light, and matches the reflectedlight wavelength to determine the observed material. For example, suchan outward facing system may detect the presence of cancerous cellsamong healthy cells. Because cancerous, or other abnormal cells, reflectand absorb light differently than healthy cells, a reflection of lightat certain wavelengths can indicate the presence and amount ofabnormality.

In some embodiments, the controller receives the captured target objectfrom the real world capturing system, and applies a label to the targetobject indicative of the identified property. In some embodiments, thelabel is a textual label or prompt within a display of the headmounted-member. In some embodiments, the label is an audio prompt to auser. In some embodiments, the label is a virtual image of similartissue, such as referenced in a medical book, superimposed near thetarget object for ready comparative analysis by the user.

In some embodiments, the head-mounted member may comprise an eyeglassesframe. The eyeglasses frame may be a binocular eyeglasses frame. The oneor more radiation emitters may comprise a light source, such as a lightemitting diode. The one or more radiation emitters may comprise aplurality of light sources configured to emit electromagnetic radiationat two or more different wavelengths. The plurality of light sources maybe configured to emit electromagnetic radiation at a first wavelength ofabout 660 nanometers, and a second wavelength of about 940 nanometers.The one or more radiation emitters may be configured to emitelectromagnetic radiation at the two different wavelengths sequentially.The one or more radiation emitters may be configured to emitelectromagnetic radiation at the two predetermined wavelengthssimultaneously. The one or more electromagnetic radiation detectors maycomprise a device selected from the group consisting of: a photodiode, aphotodetector, and a digital camera sensor. The one or moreelectromagnetic radiation detectors may be positioned and oriented toreceive light reflected after encountering a target object. The one ormore electromagnetic radiation detectors may be positioned and orientedto receive light reflected after encountering observed tissue ormaterial; that is, the one or more electromagnetic radiation detectorsare oriented substantially in the same direction as the one or moreelectromagnetic radiation emitters, whether inward facing towards auser's eye or outward facing towards a user's environment.

The controller may be further configured to cause the plurality of lightsources to emit a cyclic pattern of first wavelength on, then secondwavelength on, then both wavelengths off, such that the one or moreelectromagnetic radiation detectors detect the first and secondwavelengths separately. The controller may be configured to cause theplurality of light emitting diodes to emit a cyclic pattern of firstwavelength on, then second wavelength on, then both wavelengths off, ina cyclic pulsing pattern about thirty times per second.

In some embodiments, the controller may be configured to calculate aratio of first wavelength light measurement to second wavelength lightmeasurement. In some embodiments this ratio may be further converted toan oxygen saturation reading via a lookup table based at least in partupon the Beer-Lambert law. In some embodiments, the ratio is convertedto a material identifier in external lookup tables, such as stored in anabsorption database module on a head-mounted member or coupled to ahead-mounted member on a local or remote processing module. For example,an absorption database module for absorption ratios or wavelengthreflection of particular tissues may be stored in a “cloud” storagesystem accessible by health care providers and accessed through a remoteprocessing module. In some embodiments, an absorption database modulemay store absorption properties (such as wavelength ratios or wavelengthreflections) for certain foods and be permanently stored on a localprocessing module to the head-mounted member.

In this way, the controller may be configured to operate the one or moreelectromagnetic radiation emitters and one or more electromagneticradiation detectors to function as a broad use head-mountedspectroscope. The controller may be operatively coupled to an opticalelement coupled to the head-mounted member and viewable by the user,such that the output of the controller indicating the wavelengthproperties indicative of a particular tissue property or materialotherwise may be viewed by the user through the optical element. The oneor more electromagnetic radiation detectors may comprise a digital imagesensor comprising a plurality of pixels, wherein the controller isconfigured to automatically detect a subset of pixels which arereceiving the light reflected after encountering, for example, tissue orcells within the tissue. In some embodiments, such subset of pixels areused to produce an output representative of the target object within thefield of view of the digital image sensor. For example, the output maybe a display label that is indicative of an absorption level of thetissue. In some embodiments, comparative values are displayed as anoutput. For example, an output may be a percentage saturation of oxygenof blood from a first analysis time and a percentage saturation ofoxygen at a second analysis time with a rate of change noted between thetwo times. In these embodiments, ailments such as diabetic retinopathymay be detected by recognizing changes in measured properties over time.

In some embodiments, the controller may be configured to automaticallydetect the subset of pixels based at least in part upon reflected lightluminance differences amongst signals associated with the pixels. Thecontroller may be configured to automatically detect the subset ofpixels based at least in part upon reflected light absorptiondifferences amongst signals associated with the pixels. In suchembodiments, such subsets may be isolated pixels and flagged for furtheranalysis, such as additional irradiation or mapping, or a virtual imagemay be overlaid on such pixels to provide visual contrast to theisolated pixels displaying other properties to serve as a notice to auser of the different properties of the subpixels identified by thesystem.

In some embodiments, the system data collection is time multiplexed notonly for pulsing and recording light pulses, but passively collected atmultiple times a day. In some embodiments, a GPS or other similarmapping system is coupled to the system to correlate a user's locationor time of day with certain physiological data collected. For example, auser may track physiological responses relative to certain locations oractivities throughout a day.

These and many other features and advantages of the present inventionwill be appreciated when the following figures and description arefurther taken into account.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates certain aspects of an augmented reality systempresentation to a user.

FIGS. 2A-2D illustrate certain aspects of various augmented realitysystems for wearable computing applications, featuring a head-mountedcomponent operatively coupled to local and remote process and datacomponents.

FIG. 3 illustrates certain aspects of a connectivity paradigm between awearable augmented or virtual reality system and certain remoteprocessing and/or data storage resources.

FIGS. 4A-4D illustrate various aspects of pulse oximetry configurationsand calibration curves related to scattering of light in oxygenation ofblood.

FIG. 5 illustrates a head-mounted spectroscopy system integrating AR/VRfunctionality according to some embodiments.

FIG. 6 illustrates various aspects of a wearable AR/VR system featuringintegrated spectroscopy modules according to some embodiments.

FIGS. 7A-7B are an example light saturation curve chart indicative ofselect properties by wavelengths.

FIG. 8 illustrates a method for identifying materials or materialproperties through a head-mounted spectroscopy system according to someembodiments.

DETAILED DESCRIPTION

Some AR and VR systems comprise a processing capability, such as acontroller or microcontroller, and also a power supply to power thefunction of the various components, and by virtue of the fact that atleast some of the components in a wearable computing system, such as anAR or VR system, are close to the body of the user operating them, thereis an opportunity to utilize some of these system components to conductcertain physiologic monitoring relative to the user. For example,physiologic monitoring may be conducted by measuring light absorption.

In conventional light absorption measurement techniques (for examplepulse oximetry meters attachable to a person's finger as in FIG. 4A orin glucose detection), light is emitted in a controlled and fixeddirection and received in a controlled and fixed receiver. Light ispulsed at different wavelengths through surrounding tissue structureswhile also being detected at another side of the tissue structure (andtherefore measuring light properties such as absorption and scatter). Insuch systems, the measurement of light emitted compared to themeasurement of light detected can provide an output that is proportionalto, or reads as, an estimated tissue or tissue property (for example, anestimated blood oxygen saturation level for pulse oximetry meters), orsimply a material or tissue type otherwise. Calibration curves depictinga ratio of light of interest relative to other light are also possibleto predict properties of underlying tissue as a function of the lightincident to it as shown in FIG. 4D.

Raman spectroscopy is another technique that measures inelasticscattering of photons released by irradiated molecules. Specificmolecules will present specific shifts of wavelengths when irradiated,thereby presenting unique scattering effects that may be used to measureand quantify molecules within a sample.

FIG. 4B illustrates a chart of the absorption spectra of hemoglobin thatis oxygenated (806) versus deoxygenated (808), and as shown in suchplots (806, 808), in the red light wavelength range of theelectromagnetic spectrum, such as around 660 nm, there is a notabledifference in absorption for oxygenated versus deoxygenated hemoglobin,whereas there is an inverted difference at around 940 nm in the infraredwavelength range. Pulsing radiation at such wavelengths and detectingwith a pulse oximeter is known to take advantage of such absorptiondifferences in the determination of oxygen saturation for a particularuser.

While pulse oximeters (802) typically are configured to at leastpartially encapsulate a tissue structure such as a finger (804) or earlobe, certain desktop style systems have been suggested, such as that(812) depicted in FIG. 4C, to observe absorption differences in vesselsof the eye, such as retinal vessels, but may be configured to detectproperties of other tissues as well.

Such a configuration (812) may be termed a flow oximeter or spectroscopesystem and may comprise components as shown, including a camera (816),zoom lens (822), first (818) and second (820) light emitting diodes(LEDs), and one or more beam splitters (814). While it would be valuableto certain users, such as high-altitude hikers, athletes, or personswith certain cardiovascular or respiratory problems, to be able toretrieve information of their blood oxygen saturation as they move abouttheir day and conduct their activities, or for caregivers to analyzetissue in real time for underlying abnormalities, most configurationsinvolve a somewhat inconvenient encapsulation of a tissue structure, orare not portable or wearable, do not consider other absorptionproperties indicative of other tissue states or materials, or do notcorrelate gaze a user is looking at as part of directionality of itssensors (in other words, selectivity of target objects of foridentification and analysis by spectroscopy is lacking).

Advantageously, in some embodiments, a solution is presented hereinwhich combines the convenience of wearable computing in the form of anAR or VR system with an imaging means to determine additional tissueidentification and properties in real time within a field of view of auser.

Referring to FIGS. 2A-2D, some general componentry options areillustrated. In the portions of the detailed description which followthe discussion of FIGS. 2A-2D, various systems, subsystems, andcomponents are presented for addressing the objectives of providing ahigh-quality, comfortably-perceived display system for human VR and/orAR that access and create external information sources.

As shown in FIG. 2A, an AR system user (60) is depicted wearing headmounted component (58) featuring a frame (64) structure coupled to adisplay system (62) positioned in front of the eyes of the user. Aspeaker (66) is coupled to the frame (64) in the depicted configurationand positioned adjacent the ear canal of the user (in one embodiment,another speaker, not shown, is positioned adjacent the other ear canalof the user to provide for stereo/shapeable sound control). The display(62) is operatively coupled (68), such as by a wired lead or wirelessconnectivity, to a local processing and data module (70) which may bemounted in a variety of configurations, such as fixedly attached to theframe (64), fixedly attached to a helmet or hat (80) as shown in theembodiment of FIG. 2B, embedded in headphones, removably attached to thetorso (82) of the user (60) in a backpack-style configuration as shownin the embodiment of FIG. 2C, or removably attached to the hip (84) ofthe user (60) in a belt-coupling style configuration as shown in theembodiment of FIG. 2D.

The local processing and data module (70) may comprise a processor orcontroller (e.g., a power-efficient processor or controller), as well asdigital memory, such as flash memory, both of which may be utilized toassist in the processing, caching, and storage of data a) captured fromsensors which may be operatively coupled to the frame (64), such aselectromagnetic emitters and detectors, image capture devices (such ascameras), microphones, inertial measurement units, accelerometers,compasses, GPS units, radio devices, and/or gyros; and/or b) acquiredand/or processed using the remote processing module (72) and/or remotedata repository (74), possibly for passage to the display (62) aftersuch processing or retrieval. The local processing and data module (70)may be operatively coupled (76, 78), such as via a wired or wirelesscommunication links, to the remote processing module (72) and remotedata repository (74) such that these remote modules (72, 74) areoperatively coupled to each other and available as resources to thelocal processing and data module (70).

In one embodiment, the remote processing module (72) may comprise one ormore relatively powerful processors or controllers configured to analyzeand process data, light properties emitted or received, and/or imageinformation. In one embodiment, the remote data repository (74) maycomprise a relatively large-scale digital data storage facility, whichmay be available through the internet or other networking configurationin a “cloud” resource configuration. In one embodiment, all data isstored and all computation is performed in the local processing and datamodule, allowing fully autonomous use from any remote modules.

Referring now to FIG. 3 , a schematic illustrates coordination betweenthe cloud computing assets (46) and local processing assets, which may,for example reside in head mounted components (58) coupled to the user'shead (120) and a local processing and data module (70), coupled to theuser's belt (308); therefore the component 70 may also be termed a “beltpack” 70), as shown in FIG. 3 . In one embodiment, the cloud (46)assets, such as one or more server systems (110) are operatively coupled(115), such as via wired or wireless networking (wireless generallybeing preferred for mobility, wired generally being preferred forcertain high-bandwidth or high-data-volume transfers that may bedesired), directly to (40, 42) one or both of the local computingassets, such as processor and memory configurations, coupled to theuser's head (120) and belt (308) as described above. These computingassets local to the user may be operatively coupled to each other aswell, via wired and/or wireless connectivity configurations (44), suchas the wired coupling (68) discussed below in reference to FIG. 8 .

In one embodiment, to maintain a low-inertia and small-size subsystemmounted to the user's head (120), primary transfer between the user andthe cloud (46) may be via the link between the subsystem mounted at thebelt (308) and the cloud, with the head mounted (120) subsystemprimarily data-tethered to the belt-based (308) subsystem using wirelessconnectivity, such as ultra-wideband (“UWB”) connectivity, as iscurrently employed, for example, in personal computing peripheralconnectivity applications.

With efficient local and remote processing coordination, and anappropriate display device for a user, such as the user interface oruser display system (62) shown in FIG. 2A, or variations thereof,aspects of one world pertinent to a user's current actual or virtuallocation may be transferred or “passed” to the user and updated in anefficient fashion. In other words, a map of the world may be continuallyupdated at a storage location which may, e.g., partially reside on theuser's AR system and partially reside in the cloud resources. The map(also referred to as a “passable world model”) may be a large databasecomprising raster imagery, 3-D and 2-D points, parametric informationand other information about the real world. As more and more AR userscontinually capture information about their real environment (e.g.,through cameras, sensors, IMUs, etc.), the map becomes more and moreaccurate and complete.

With a configuration as described above, wherein there is one worldmodel that can reside on cloud computing resources and be distributedfrom there, such world can be “passable” to one or more users in arelatively low bandwidth form preferable to trying to pass aroundreal-time video data or the like. In some embodiments, the augmentedexperience of the person standing near the statue (i.e., as shown inFIG. 1 ) may be informed by the cloud-based world model, a subset ofwhich may be passed down to them and their local display device tocomplete the view. A person sitting at a remote display device, whichmay be as simple as a personal computer sitting on a desk, canefficiently download that same section of information from the cloud andhave it rendered on their display. Indeed, one person actually presentin the park near the statue may take a remotely-located friend for awalk in that park, with the friend joining through virtual and augmentedreality. The system will need to know where the street is, where thetrees are, where the statue is—but with that information on the cloud,the joining friend can download from the cloud aspects of the scenario,and then start walking along as an augmented reality local relative tothe person who is actually in the park.

3-D points may be captured from the environment, and the pose (i.e.,vector and/or origin position information relative to the world) of thecameras that capture those images or points may be determined, so thatthese points or images may be “tagged”, or associated, with this poseinformation. Then points captured by a second camera may be utilized todetermine the pose of the second camera. In other words, one can orientand/or localize a second camera based upon comparisons with taggedimages from a first camera. Then this knowledge may be utilized toextract textures, make maps, and create a virtual copy of the real world(because then there are two cameras around that are registered).

So, at the base level, in some embodiments a person-worn system may beutilized to capture both 3-D points and the 2-D images that produced thepoints, and these points and images may be sent out to a cloud storageand processing resource. They may also be cached locally with embeddedpose information (e.g., cache the tagged images); so, the cloud may haveon the ready (e.g, in available cache) tagged 2-D images (e.g., taggedwith a 3-D pose), along with 3-D points. If a user is observingsomething dynamic (e.g., a scene with moving objects or features),he/she may also send additional information up to the cloud pertinent tothe motion (for example, if looking at another person's face, the usercan take a texture map of the face and push that up at an optimizedfrequency even though the surrounding world is otherwise basicallystatic). As noted above, more information on object recognizers and thepassable world model may be found in U.S. patent application Ser. No.14/205,126, entitled “System and method for augmented and virtualreality”, which is incorporated by reference in its entirety herein,along with the following additional disclosures, which relate toaugmented and virtual reality systems such as those developed by MagicLeap, Inc. of Fort Lauderdale, Fla.: U.S. patent application Ser. No.14/641,376; U.S. patent application Ser. No. 14/555,585; U.S. patentapplication Ser. No. 14/212,961; U.S. patent application Ser. No.14/690,401; U.S. patent application Ser. No. 13/663,466; U.S. patentapplication Ser. No. 13/684,489; and U.S. Patent Application Ser. No.62/298,993, each of which is incorporated by reference herein in itsentirety.

In some embodiments, the use of such passable world information maypermit identification and labelling of objects by spectroscopy to thenpass between users. For example, in a clinical setting, a firstcaregiver operating a device implementing features of the presentdisclosure may map and detect cancerous tissue on a patient and assignand apply a virtual label, much like a metatag, to the tissue. A secondcaregiver similarly wearing such a device may then look at the samecancerous tissue cell cluster and receive notice of the virtual labelidentifying such cells without needing to engage in one or more ofemitting light, receiving light, matching an absorption trait to atissue, and labeling the tissue independently.

GPS and other localization information may be utilized as inputs to suchprocessing. It will be appreciated that highly accurate localization ofthe user's head, totems, hand gestures, haptic devices etc. canfacilitate displaying appropriate virtual content to the user, orpassable virtual or augmented content among users in a passable world.

Referring to FIG. 5 , a top orthogonal view of a head mountablecomponent (58) of a wearable computing configuration is illustratedfeaturing various integrated components for an exemplary spectroscopysystem. The configuration features two display elements(62—binocular—one for each eye), two forward-oriented cameras (124) forobserving and detecting the world around the user, each camera (124)having an associated field of view (18, 22), and at least onespectroscopy array (126, described in greater detail in FIG. 6 ), with afield of view (20); also a forward-oriented relatively high resolutionpicture camera (156) with a field of view (26), one or more inertialmeasurement units (102), and a depth sensor (154) with an associatedfield of view (24), such as described in the aforementioned incorporatedby reference disclosures. Facing toward the eyes (12, 13) of the userand coupled to the head mounted component (58) frame are eye trackingcameras (828, 830) and inward emitters and receivers (832, 834). One ofskill in the art will appreciate the inward emitters and receivers (832,834) emit and receive light directed towards the eyes in irradiationpattern (824, 826) much in the same way spectroscopy array (126) doesfor outward objects in its field of view (20). These components, orcombinations less inclusive of all components are operatively coupledsuch as by wire lead, to a controller (844), which is operativelycoupled (848) to a power supply (846), such as a battery.

In some embodiments, the display elements (62) include one or morewaveguides (e.g., a waveguide stack) which are optically transmissiveand allow the user to “see” the world by receiving light from the world.The waveguides also receive light containing display information andpropagate and eject the light to the user's eyes (12, 13), to therebydisplay an image to the user. Preferably, light propagating out of thewaveguide provides particular, defined levels of wavefront divergencecorresponding to different depth planes (e.g., the light forming animage of an object at a particular distance from the user has awavefront divergence that corresponds to or substantially matches thewavefront divergence of light that would reach the user from that objectif real). For example, the waveguides may have optical power and may beconfigured to output light with selectively variable levels of wavefrontdivergence. It will be appreciated that this wavefront divergenceprovides cues to accommodation for the eyes (12, 13). In addition, thedisplay elements (62) utilize binocular disparity to further providedepth cues, e.g. cues to vergence of the eyes (12, 13). Advantageously,the cues to accommodation and cues to vergence may match, e.g., suchthat they both correspond to an object at the same distance from theuser. This accommodation-vergence matching facilitates the long-termwearability of a system utilizing the head-mounted member (58).

With continued reference to FIG. 5 , preferably, each emitter (126, 832,834) is configured to controllably emit electromagnetic radiation in twoor more wavelengths, such as about 660 nm, and about 940 nm, such as byLEDs, and preferably the fields of irradiation (824, 826) are orientedto irradiate targeted objects or surfaces. In some embodiments, targetedobjects are inward, such as eyes (12, 13) and irradiation patterns (824,826) may be fixed or broadened/narrowed to target specific areas of aneye in response to an eye tracking camera data point. In someembodiments, targeted objects are outward (e.g., away from the user),and the irradiation pattern within the field of view (20) ofspectroscope array (126) conforms to a gaze of the eyes (12, 13)determined from eye tracking cameras (828, 830).

In some embodiments, the gaze may be understood to be a vector extendingfrom the user's eye, such as extending from the fovea through the lensof the eye, and the emitters (832, 834) may output infrared light on theuser's eyes, and reflections from the eye (e.g., corneal reflections)may be monitored. A vector between a pupil center of an eye (e.g., thedisplay system may determine a centroid of the pupil, for instancethrough infrared imaging) and the reflections from the eye may be usedto determine the gaze of the eye. In some embodiments, when estimatingthe position of the eye, since the eye has a sclera and an eyeball, thegeometry can be represented as two circles layered on top of each other.The eye pointing vector may be determined or calculated based on thisinformation. Also the eye center of rotation may be estimated since thecross section of the eye is circular and the sclera swings through aparticular angle. This may result in a vector distance because ofautocorrelation of the received signal against known transmitted signal,not just ray traces. The output may be seen as a Purkinje image 1400which may in turn be used to track movement of the eyes.

One of skill in the art will appreciate other ways to determine anirradiation pattern within field of view (20) such as by head poseinformation determined by one or more of IMU (102).

In some embodiments, the emitters may be configured to emit wavelengthssimultaneously, or sequentially, with controlled pulsatile emissioncycling. The one or more detectors (126, 828, 830) may comprisephotodiodes, photodetectors, and/or digital camera sensors, andpreferably are positioned and oriented to receive radiation that hasencountered the targeted tissue or material or object otherwise. The oneor more electromagnetic radiation detectors (126, 828, 830) may comprisea digital image sensor comprising a plurality of pixels, wherein thecontroller (844) is configured to automatically detect a subset ofpixels which are receiving the light reflected after encountering atarget object, and to use such subset of pixels to produce an output.

In some embodiments, the output is a function of matching received lightagainst emitted light to a target from an absorption database ofmaterials and material properties. For example, in some embodiments, anabsorption database comprises a plurality of absorption charts such asdepicted in FIGS. 7A and 7B. It will be appreciated that a databasecomprising charts may include electronic representations ortransformations of the information in the charts, and the use of theterm charts herein includes such representations or transformations.FIGS. 7A and 7B is merely used as an example, but demonstrates varioustissue properties that may be detected from a given system emittinglight from a particular light source and receiving light of a particularwavelength and/or light property to determine the probability of anobserved target being a particular tissue or having particularproperties within the tissue. Other charts, such as either saturationcurves or calibration curves, may be selectively accessed by a user. Forexample, a user could choose absorption databases for a particular lightsource or wavelength patterns and then look around until thespectroscopy system identifies material matching the propertiesrequested. Such an embodiment may be termed a “closed search,” or onethat looks for specific properties as opposed to an “open search” thatlooks at any target and then searches databases for matches on the lightproperties detected.

The controller (844) may be configured to automatically detect a subsetof pixels within a field of view (124, or 126, or 824, 826, FIG. 5 )based at least in part upon reflected light properties differencesamongst signals associated with the pixels. For example, the controller(844) may be configured to automatically detect the subset of pixelsbased at least in part upon reflected light absorption differencesamongst signals associated with the pixels. Without being limited bytheory, light impacting upon an object will reflect, transmit (absorb),or scatter upon striking the object, such that R+T+S=1 (withR=reflection from the object, T=transmission/absorption into the object,and S=scatter from the object). If a particular subset of pixelsreflects a higher proportion of light relative to surrounding subpixels,the controller may isolate these subpixels or note or register the pixellocation for these different properties in a memory system. In someembodiments, the pixel location are stored in a passable world mappingsystem as dense or sparse mapping points such as additional users of ahead mounted display system access the map, the subset of pixels arepassed to the additional user and accessed and/or displayed on thesecond user's display.

Referring to FIG. 6 , a spectroscopy array (126) may comprise a lightsource (612) emitting light (613) towards a target object (620). In someembodiments, the light source (612) is an electromagnetic emitter suchas light emitting diodes. In some embodiments, the direction of emittedlight (613) is substantially the same as a gaze orientation of a user(60) or a head pose orientation of a user (60). In some embodiments,photodetectors (614) capture reflected light (615) from the targetobject. In some embodiments, a processor (610), which may be controller(844) depicted in FIG. 5 , determines an absorption property betweenemitted light (613) and reflected light (615) and matches the propertyfrom absorption database (630). In some embodiments, absorption database(630) is stored on a local processing module such as module (70)depicted in FIG. 2A for example; in some embodiments, absorptiondatabase (630) is stored on remote processing module (72) such as theone depicted in FIG. 2A.

Object (620) is depicted as an apple in FIG. 6 for simplicity, andthough food properties have their respective light absorption propertiesand embodiments of the invention may be used to identify food by itslight properties, more sophisticated uses are also envisioned. In someembodiments, outward facing spectroscopy array (126) identifies tissuesource (624), e.g., an arm as depicted for illustrative purposes.Emitted light (613) may impact upon tissue source (624) and reflectedlight (615) may indicate the presence of irregular cells (626) amongstregular cells (625). As light source (612) irradiates tissue source(624), irregular cells (626) will return a different light property tophotodetectors (614) than regular cells (625). Irregular cells (626) maybe cancerous, be part of scar tissue, or even healthy cells amongst thetissue simply indicating or having a difference with surrounding cells,for example indicating where blood vessels or bone within tissue source(624) may be located. In some embodiments, regular cells constitute themajority of cells in a sample under analysis and irregular cellsconstitute a minority of the cells of the sample, the irregular cellsexhibiting a different detectable property than the regular cells. Insome embodiments, real world cameras capturing images on a pixel levelmay mark such irregular cells (626). As previously described, one suchmarking may be a labeling system applying a textual image proximate tothe irregular cells (626), another such labeling system may be a coloroverlay onto irregular cells (626), as seen through the display element62 (FIG. 5 ).

Thus, with reference again to FIG. 5 , a system is presented fordetermining tissue properties or materials otherwise through a wearablecomputing system, such as one for AR or VR, comprising: a head-mountedmember (58) removably coupleable to the user's head; one or moreelectromagnetic radiation emitters (126, 832, 834) coupled to thehead-mounted member (58) and configured to emit light with at least twodifferent wavelengths in inward directions or outwards directions, oneor more electromagnetic radiation detectors (126, 828, 830) coupled tothe head-mounted member and configured to receive light reflected afterencountering a target object; and a controller (844) operatively coupledto the one or more electromagnetic radiation emitters (126, 832, 834)and one or more electromagnetic radiation detectors (126, 828, 830) andconfigured to cause the one or more electromagnetic radiation emittersto emit pulses of light while also causing the one or moreelectromagnetic radiation detectors to detect levels of light absorptionrelated to the emitted pulses of light, and to produce a displayableoutput.

The head-mounted member (58) may comprise frame configured to fit on theuser's head, e.g., an eyeglasses frame. The eyeglasses frame may be abinocular eyeglasses frame; alternative embodiments may be monocular.The one or more emitters (126, 832, 834) may comprise a light source,for example at least one light emitting diode or other electromagneticradiation emitter, emitting light at multiple wavelengths. The pluralityof light sources may be configured to preferably emit at two wavelengthsof light, e.g., a first wavelength of about 660 nanometers, and a secondwavelength of about 940 nanometers.

In some embodiments, the one or more emitters (126, 832, 834) may beconfigured to emit light at the respective wavelengths sequentially. Insome embodiments, the one or more emitters (126, 832, 834) may beconfigured to emit light at the respective wavelengths simultaneously.The one or more electromagnetic radiation detectors (126, 828, 830) maycomprise a device selected from the group consisting of: a photodiode, aphotodetector, and a digital camera sensor. The controller (844) may befurther configured to cause the plurality of light emitting diodes toemit a cyclic pattern of first wavelength on, then second wavelength on,then both wavelengths off, such that the one or more electromagneticradiation detectors detect the first and second wavelengths separately.The controller (844) may be configured to cause the plurality of lightemitting diodes to emit a cyclic pattern of first wavelength on, thensecond wavelength on, then both wavelengths off, in a cyclic pulsingpattern about thirty times per second. The controller (844) may beconfigured to calculate a ratio of first wavelength light measurement tosecond wavelength light measurement, and wherein this ratio is convertedto an oxygen saturation reading via a lookup table based at least inpart upon the Beer-Lambert law.

The controller (844) may be configured to operate the one or moreemitters (126, 832, 834) and one or more electromagnetic radiationdetectors (126, 828, 830) to function as a head-mounted spectroscope.The controller (844) may be operatively coupled to an optical element(62) coupled to the head-mounted member (58) and viewable by the user,such that the output of the controller (844) that is indicative of aparticular material property or tissue property may be viewed by theuser through the optical element (62).

FIG. 7A is an example light property absorption chart that may bereferenced by an absorption database (630, FIG. 6 ). As depicted,various light source types, such as IR, NIR, or light emitting diodes inthe visible spectrum may be optimal for detecting certain tissues andproperties within the tissue. In some embodiments, an absorption ratioor scatter in calibration curve is computed from emitted light toreflected light and applied to the given absorption database (630) suchas depicted in FIG. 7A to determine the underlying tissue and/orproperties within or determine abnormalities.

FIG. 7B depicts potential “overlap” of wavelengths. As depicted,“oxygenated blood” may overlap with “deoxygenated blood” at certainwavelengths, muting the results that a spectroscopic processes mayprovide. To avoid this potential overlap, in some embodiments, light ata second different wavelength is emitted to provide a second source oflight to measure and compare.

FIG. 8 illustrates a method (850) for using a wearable AR/VR systemfeaturing spectroscopy components to identify tissue or propertieswithin tissue. Method (850) begins at (851) with the system orientinglight sources to a target object. In some embodiments, the orienting haslight sources directed inwards towards the eyes of a user, and may befixed or scanning the eye such as scanning the retina. In someembodiments, the orienting is by determining an eye gaze or head pose ofthe user and orienting a light source in substantially the samedirection towards a target object within such gaze or pose field ofview, or towards feature landmarks or target objects.

In some embodiments, at (852) light sources emit light in an irradiationpattern towards the target object or surface. In some embodiments, thelight is pulsed at timed intervals by a timer. In some embodiments, thelight source emits light of at least one wavelength and at (854)radiation detectors, such as photo detectors, receive reflected light.In some embodiments, the detectors are also operatively coupled to atimer to indicate if received light was initially pulsed at a certaintime to determine changes in light properties upon reflecting on thetarget object. In some embodiments, (852) begins concurrent with mappingat (853) but this sequence is not necessarily so.

In some embodiments, real world capturing systems may begin to map thetarget object at (853). In some embodiments, such mapping may includereceiving passable world data of the target object. In some embodiments,mapping may include depth sensor analysis of the contours of the targetobject. In some embodiments, mapping may include building a mesh modelof the items within the field of view and referencing them for potentiallabeling. In some embodiments, the target object is not a specificobject within the field of view that may be captured by a depth sensor,but rather is a depth plane within the field of view itself.

In some embodiments, at (855) a controller analyzes the emitted lightcompared to the received light, such as under the Beer-Lambert law orthe optical density relationship (described below) or scatter pattern ofa calibration curve. In some embodiments, at (856) the compared lightproperties are referenced in an absorption database, either locallystored on the system or remotely accessed through the system, toidentify the tissue or tissue property of the target object. In someembodiments, an absorption database may comprise saturation lightcharts, such as the one depicted in FIG. 4B, or may comprise calibrationcurves of particular light wavelengths.

In some embodiments, at (854) the radiation detectors do not receivelight of different wavelengths than the wavelength of the light emittedat (852), and a controller cannot conduct a spectroscopic analysis. Suchan occasion would occur as in FIG. 7B, with overlap of wavelengths incertain ranges for oxygenated and deoxygenated blood. In someembodiments, at (854 a) no wavelength difference is detected between theemitted light and received light, and substep (854 b) initiates byemitting light at another different wavelength than that emitted at(852). The new light emitted and light received information is thendelivered to a controller at (855).

In some embodiments, real world cameras may additionally, subsequent tomapping a target object (853) and potentially concurrent with each of(852 through 856), identify subpixels within a field of field indicativeof irregularities at (857). For example, in some embodiments, colorcontrast between pixels is detected during real world capture at (853)and at (857) these pixels are further altered to highlight such contrastas potential unhealthy cells. In some embodiments, real world capture(853) detects irregular lines among pixel clusters and at (857) thepixels bounded by the irregular lines are marked (such as by a virtualcolor overlay) on a user display.

In some embodiments, method (850) terminates at (858) with the systemdisplaying the tissue or material property of the tissue to the user. Insome embodiments, display may comprise a textual label virtuallydisplayed proximate to the target object, an audio label describing thetarget object as determined from the absorption database (630), or avirtual image of similar tissue or object identified by absorptiondatabase (630) juxtaposed proximate to the target object.

In some embodiments, a significant amount of the spectroscopy activityis implemented with software operated by the controller (844), such thatan initial task of locating desired targets (e.g., blood vessels, muscletissue, bone tissue, or other tissue and at a desired depth) isconducted using digital image processing (such as by color, grayscale,and/or intensity thresholding analysis using various filters. Suchtargeting may be conducted using pattern, shape recognition or texturerecognition. Cancerous cells or otherwise irregular cells commonly haveirregular borders. A camera system may identify a series of pixelswithin a camera field of view (such as cameras 124 and field of view 18,22 of FIG. 5 ) with an irregular, non-linear pattern and promptattention to identify such as a border to a potentially unhealthy cell.Alternatively, the software and controller may be configured to use theintensity of the center of the targeted object and the intensity of thesurrounding objects/tissue to determine contrast/optical density withthe targeted object to determine abnormalities. Such measures may merelybe used to identify areas of interest for spectroscopic scan consistentwith this disclosure, and not necessarily a means of identifying tissueitself. Further, as previously described with reference to irregularcells (626) in FIG. 6 , an augmented reality system may overlay a labelor color pattern within the borders of the potentially unhealthy cellsto flag them/highlight them against surrounding healthy cells.

In some embodiments, the controller (844) may be utilized to calculatedensity ratios (contrast) and to calculate the oxygen saturation fromthe density ratios of various pulse oximetry properties in bloodvessels. Vessel optical density (“O.D.”) at each of the two or moreemitted wavelengths may be calculated using the formula:ODvessel=−log₁₀(Iv/It)

wherein ODvessel is the optical density of the vessel; Iv is the vesselintensity; and It is the surrounding tissue intensity.

Oxygen saturation (also termed “SO2”) in a blood vessel may becalculated as a linear ratio of vessel optical densities (OD ratio, or“ODR”) at the two wavelengths, such that:SO₂=ODR=OD_(firstwavelength)/OD_(secondwavelength)

In one embodiment, wavelengths of about 570 nm (sensitive todeoxygenated hemoglobin) and about 600 nm (sensitive to oxygenatedhemoglobin) may be utilized in vessel oximetry, such thatSO2=ODR=OD_(600nm)/OD570 nm; such formula does not account for adjustingthe ratio by a calibration coefficient.

The above formulas are merely examples of references for calculatingmaterial properties. One of skill in the art will appreciate many othertissue properties and relationships a controller may determine.

It will be appreciated that utilizing the controller (844) to performcalculations and/or make determinations may involve performingcalculations locally on a processor within the controller (844). In someother embodiments, performing calculations and/or making determinationswith the controller (844) may involve utilizing the controller tointerface with external computing resources, e.g., resources in thecloud (46) such as servers (110).

Computer Vision

As discussed above, the spectroscopy system may be configured to detectobjects in or features (e.g. properties) of objects in the environmentsurrounding the user. In some embodiments, objects or properties ofobjects present in the environment may be detected using computer visiontechniques. For example, as disclosed herein, the spectroscopy system'sforward-facing camera may be configured to image an object and thesystem may be configured to perform image analysis on the images todetermine the presence of features on the objects. The system mayanalyze the images, absorption determinations, and/or reflected and/orscattered light measurements acquired by the outward-facing imagingsystem to object recognition, object pose estimation, learning,indexing, motion estimation, or image restoration, etc. One or morecomputer vision algorithms may be selected as appropriate and used toperform these tasks. Non-limiting examples of computer vision algorithmsinclude: Scale-invariant feature transform (SIFT), speeded up robustfeatures (SURF), oriented FAST and rotated BRIEF (ORB), binary robustinvariant scalable keypoints (BRISK), fast retina keypoint (FREAK),Viola-Jones algorithm, Eigenfaces approach, Lucas-Kanade algorithm,Horn-Schunk algorithm, Mean-shift algorithm, visual simultaneouslocation and mapping (vSLAM) techniques, a sequential Bayesian estimator(e.g., Kalman filter, extended Kalman filter, etc.), bundle adjustment,Adaptive thresholding (and other thresholding techniques), IterativeClosest Point (ICP), Semi Global Matching (SGM), Semi Global BlockMatching (SGBM), Feature Point Histograms, various machine learningalgorithms (such as e.g., support vector machine, k-nearest neighborsalgorithm, Naive Bayes, neural network (including convolutional or deepneural networks), or other supervised/unsupervised models, etc.), and soforth.

As discussed herein, the objects or features (including properties) ofobjects may be detected based on one or more criteria (e.g., absorbance,light reflection, and/or light scattering at one or more wavelengths).When the spectroscopy system detects the presence or absence of thecriteria in the ambient environment using a computer vision algorithm orusing data received from one or more sensor assemblies (which may or maynot be part of the spectroscopy system), the spectroscopy system maythen signal the presence of the object or feature.

One or more of these computer vision techniques may also be usedtogether with data acquired from other environmental sensors (such as,e.g., microphone, GPS sensor) to detect and determine various propertiesof the objects detected by the sensors.

Machine Learning

A variety of machine learning algorithms may be used to learn toidentify the presence of objects or features of objects. Once trained,the machine learning algorithms may be stored by the spectroscopysystem. Some examples of machine learning algorithms may includesupervised or non-supervised machine learning algorithms, includingregression algorithms (such as, for example, Ordinary Least SquaresRegression), instance-based algorithms (such as, for example, LearningVector Quantization), decision tree algorithms (such as, for example,classification and regression trees), Bayesian algorithms (such as, forexample, Naive Bayes), clustering algorithms (such as, for example,k-means clustering), association rule learning algorithms (such as, forexample, a-priori algorithms), artificial neural network algorithms(such as, for example, Perceptron), deep learning algorithms (such as,for example, Deep Boltzmann Machine, or deep neural network),dimensionality reduction algorithms (such as, for example, PrincipalComponent Analysis), ensemble algorithms (such as, for example, StackedGeneralization), and/or other machine learning algorithms. In someembodiments, individual models may be customized for individual datasets. For example, the wearable device may generate or store a basemodel. The base model may be used as a starting point to generateadditional models specific to a data type (e.g., a particular user), adata set (e.g., a set of absorbance, light reflection, and/or lightscattering values obtained at one or more wavelengths), conditionalsituations, or other variations. In some embodiments, the spectroscopysystem may be configured to utilize a plurality of techniques togenerate models for analysis of the aggregated data. Other techniquesmay include using pre-defined thresholds or data values.

The criteria for detecting an object or feature of an object may includeone or more threshold conditions. If the analysis of the data acquiredby a sensor (e.g., a camera or photodetector) indicates that a thresholdcondition is passed, the spectroscopy system may provide a signalindicating the detection the presence of the object in the ambientenvironment. The threshold condition may involve a quantitative and/orqualitative measure. For example, the threshold condition may include ascore or a percentage associated with the likelihood of the objectand/or feature being present. The spectroscopy system may compare thescore calculated from the sensor's data with the threshold score. If thescore is higher than the threshold level, the spectroscopy system maysignal detection of the presence of an object or object feature. In someother embodiments, the spectroscopy system may signal the absence of theobject or feature if the score is lower than the threshold.

It will be appreciated that each of the processes, methods, andalgorithms described herein and/or depicted in the figures may beembodied in, and fully or partially automated by, code modules executedby one or more physical computing systems, hardware computer processors,application-specific circuitry, and/or electronic hardware configured toexecute specific and particular computer instructions. A code module maybe compiled and linked into an executable program, installed in adynamic link library, or may be written in an interpreted programminglanguage. In some embodiments, particular operations and methods may beperformed by circuitry that is specific to a given function. In someembodiments, the code modules may be executed by hardware in thecontroller (844) (FIG. 5 ) and/or in the cloud (46) (e.g., servers(110)).

Further, certain embodiments of the functionality of the presentdisclosure are sufficiently mathematically, computationally, ortechnically complex that application-specific hardware or one or morephysical computing devices (utilizing appropriate specialized executableinstructions) may be necessary to perform the functionality, forexample, due to the volume or complexity of the calculations involved orto provide results substantially in real-time. For example, a video mayinclude many frames, with each frame having millions of pixels, andspecifically programmed computer hardware is necessary to process thevideo data to provide a desired image processing task or application ina commercially reasonable amount of time.

Code modules or any type of data may be stored on any type ofnon-transitory computer-readable medium, such as physical computerstorage including hard drives, solid state memory, random access memory(RAM), read only memory (ROM), optical disc, volatile or non-volatilestorage, combinations of the same and/or the like. In some embodiments,the non-transitory computer-readable medium may be part of one or moreof the local processing and data module (70, FIG. 2C), the remoteprocessing module (72, FIG. 2D), and remote data repository (74, FIG.2D). The methods and modules (or data) may also be transmitted asgenerated data signals (e.g., as part of a carrier wave or other analogor digital propagated signal) on a variety of computer-readabletransmission mediums, including wireless-based and wired/cable-basedmediums, and may take a variety of forms (e.g., as part of a single ormultiplexed analog signal, or as multiple discrete digital packets orframes). The results of the disclosed processes or process steps may bestored, persistently or otherwise, in any type of non-transitory,tangible computer storage or may be communicated via a computer-readabletransmission medium.

Any processes, blocks, states, steps, or functionalities in flowdiagrams described herein and/or depicted in the attached figures shouldbe understood as potentially representing code modules, segments, orportions of code which include one or more executable instructions forimplementing specific functions (e.g., logical or arithmetical) or stepsin the process. The various processes, blocks, states, steps, orfunctionalities may be combined, rearranged, added to, deleted from,modified, or otherwise changed from the illustrative examples providedherein. In some embodiments, additional or different computing systemsor code modules may perform some or all of the functionalities describedherein. The methods and processes described herein are also not limitedto any particular sequence, and the blocks, steps, or states relatingthereto may be performed in other sequences that are appropriate, forexample, in serial, in parallel, or in some other manner. Tasks orevents may be added to or removed from the disclosed exampleembodiments. Moreover, the separation of various system components inthe embodiments described herein is for illustrative purposes and shouldnot be understood as requiring such separation in all embodiments. Itshould be understood that the described program components, methods, andsystems may generally be integrated together in a single computerproduct or packaged into multiple computer products.

Various exemplary embodiments of the invention are described herein.Reference is made to these examples in a non-limiting sense. They areprovided to illustrate more broadly applicable aspects of the invention.Various changes may be made to the invention described and equivalentsmay be substituted without departing from the true spirit and scope ofthe invention. In addition, many modifications may be made to adapt aparticular situation, material, composition of matter, process, processact(s) or step(s) to the objective(s), spirit or scope of the presentinvention. Further, as will be appreciated by those with skill in theart that each of the individual variations described and illustratedherein has discrete components and features which may be readilyseparated from or combined with the features of any of the other severalembodiments without departing from the scope or spirit of the presentinventions. All such modifications are intended to be within the scopeof claims associated with this disclosure.

The invention includes methods that may be performed using the subjectdevices. The methods may comprise the act of providing such a suitabledevice. Such provision may be performed by the end user. In other words,the “providing” act merely requires the end user obtain, access,approach, position, set-up, activate, power-up or otherwise act toprovide the requisite device in the subject method. Methods recitedherein may be carried out in any order of the recited events which islogically possible, as well as in the recited order of events.

Exemplary aspects of the invention, together with details regardingmaterial selection and manufacture have been set forth above. As forother details of the present invention, these may be appreciated inconnection with the above-referenced patents and publications as well asgenerally known or appreciated by those with skill in the art. The samemay hold true with respect to method-based aspects of the invention interms of additional acts as commonly or logically employed.

In addition, though the invention has been described in reference toseveral examples optionally incorporating various features, theinvention is not to be limited to that which is described or indicatedas contemplated with respect to each variation of the invention. Variouschanges may be made to the invention described and equivalents (whetherrecited herein or not included for the sake of some brevity) may besubstituted without departing from the true spirit and scope of theinvention. In addition, where a range of values is provided, it isunderstood that every intervening value, between the upper and lowerlimit of that range and any other stated or intervening value in thatstated range, is encompassed within the invention.

Also, it is contemplated that any optional feature of the inventivevariations described may be set forth and claimed independently, or incombination with any one or more of the features described herein.Reference to a singular item, includes the possibility that there areplural of the same items present. More specifically, as used herein andin claims associated hereto, the singular forms “a,” “an,” “said,” and“the” include plural referents unless specifically stated otherwise. Inother words, use of the articles allow for “at least one” of the subjectitem in the description above as well as claims associated with thisdisclosure. It is further noted that such claims may be drafted toexclude any optional element. As such, this statement is intended toserve as antecedent basis for use of such exclusive terminology as“solely,” “only” and the like in connection with the recitation of claimelements, or use of a “negative” limitation.

Without the use of such exclusive terminology, the term “comprising” inclaims associated with this disclosure shall allow for the inclusion ofany additional element—irrespective of whether a given number ofelements are enumerated in such claims, or the addition of a featurecould be regarded as transforming the nature of an element set forth insuch claims. Except as specifically defined herein, all technical andscientific terms used herein are to be given as broad a commonlyunderstood meaning as possible while maintaining claim validity.

The breadth of the present invention is not to be limited to theexamples provided and/or the subject specification, but rather only bythe scope of claim language associated with this disclosure.

What is claimed is:
 1. A spectroscopy method comprising, under controlof one or more processors of a wearable spectroscopy system: causing oneor more light sources of the wearable spectroscopy system to emit pulsesof light with at least two different wavelengths to irradiate a targetobject; receiving, from at least one electromagnetic radiation detectorof the wearable spectroscopy system, one or more signals indicatingdetected levels of light absorption related to the emitted pulses oflight and reflected light from the target object irradiated by theemitted pulses of light; determining the output based on comparing thedetected levels of light absorption with stored absorption datacomprising light absorption properties of a plurality of materials andmatching the detected levels of light absorption to one or more of theplurality of materials of the stored absorption data; and causing thewearable spectroscopy system to display, to a user wearing the wearablespectroscopy system via a head-mountable display of the wearablespectroscopy system, an output based on the detected levels of lightabsorption.
 2. The method of claim 1, wherein the absorption data isstored in a local memory connected to the wearable spectroscopy system.3. The method of claim 1, wherein the absorption data is stored in amemory of a remote computing device.
 4. The method of claim 1, whereinthe at least one outward-facing light source comprises a plurality oflight emitting diodes.
 5. The method of claim 1, wherein the at leasttwo different wavelengths comprise a first wavelength of about 660nanometers, and a second wavelength of about 940 nanometers.
 6. Themethod of claim 1, wherein the at least two wavelengths of light areemitted sequentially.
 7. The method of claim 1, wherein the at least twowavelengths of light are emitted simultaneously.
 8. The method of claim1, wherein the at least two wavelengths of light are emitted in a cyclicpattern of a first wavelength on, then a second wavelength on, then bothfirst and second wavelengths off, such that the at least oneelectromagnetic radiation detector detects the first and secondwavelengths at different times.
 9. The method of claim 1, furthercomprising calculating a ratio of first wavelength light measurement tosecond wavelength light measurement and converting the ratio to a tissueproperty.
 10. The method of claim 9, wherein the output comprises thetissue property.
 11. The method of claim 9, wherein the tissue propertycomprises at least one property selected from the group consisting of:an estimated blood saturation level, a presence of abnormal cells, and apresence of cancerous cells.
 12. The method of claim 1, wherein the atleast one electromagnetic radiation detector comprises a device selectedfrom the group consisting of a photodiode and a photodetector.
 13. Themethod of claim 1, wherein the at least one electromagnetic radiationdetector comprises a digital image sensor.
 14. The method of claim 13,wherein the digital image sensor comprises a plurality of pixels, andwherein the controller is configured to automatically detect a subset ofpixels which are receiving the light reflected after encountering apredetermined tissue property and to produce an output that displays thelocation of the subset of pixels indicating the predetermined tissueproperty.
 15. The method of claim 1, further comprising determining apose of a head of the user via an inertial measurement unit positionalsystem of the wearable spectroscopy system.
 16. The method of claim 15,wherein the one or more outward-facing light sources emit the light in adirection corresponding to the pose of the user's head.
 17. The methodof claim 1, wherein the causing the wearable spectroscopy system todisplay the output based on the detected levels of light absorptionincludes: rendering the output representative of the target object on anoptical element of head-mountable display of the wearable spectroscopysystem.
 18. The method of claim 17, wherein the rendering the output ofthe target object includes rendering a display label that is indicativeof an absorption level of the target object.
 19. The method of claim 1,further comprising, prior to the causing the one or more light sourcesof the wearable spectroscopy system to emit the pulses of light with theat least two different wavelengths to irradiate the target object:determining the target object via application of at least one of digitalimage processing or intensity thresholding analysis on an image capturedby a digital image sensor.
 20. The method of claim 19, wherein thedetermining the target object includes receiving the image captured by acamera of the wearable spectroscopy system and identifying a series ofpixels within the camera field of view having an irregular or non-linearpattern as representing the target object.